Model-Free Neural Counterfactual Regret Minimization with Bootstrap Learning

Counterfactual Regret Minimization (CFR) has achieved many fascinating results in solving large-scale Imperfect Information Games (IIGs). Neural network approximation CFR (neural CFR) is one of the promising techniques that can reduce computation and memory consumption by generalizing decision information between similar states. Current neural algorithms have to approximate cumulative regrets. However, efficient accurate a IIG still tough challenge. In this paper, new variant, Recursive (ReCFR), proposed. ReCFR, Substitute Values (RSVs) are learned used replace It proven ReCFR converge Nash equilibrium at rate (1/T). Based on model-free with bootstrap learning, ReCFR-B, Due recursive non-cumulative nature RSVs, ReCFR-B lower-variance training targets than other CFRs. Experimental show competitive state-of-the-art much lower cost.